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ABSTRACT 

Objectives: This study investigates risk of mortality 
associated with nurses' assessments of patients by 
physiological system. We hypothesise that nursing 
assessments of in-patients performed at entry correlate 
with in-hospital mortality, and those performed just 
before discharge correlate with postdischarge mortality. 
Design: Cohort study of in-hospital and postdischarge 
mortality of patients over two 1-year periods. 
Setting: An 805-bed community hospital in Sarasota, 
Florida, USA. 

Subjects: 42 302 inpatients admitted for any reason, 
excluding obstetrics, paediatric and psychiatric 
patients. 

Outcome measures: All-cause mortalities and 
mortality OR. 

Results: Patients whose entry nursing assessments, 
other than pain, did not meet minimum standards had 
significantly higher in-hospital mortality than patients 
meeting minimums; and final nursing assessments 
before discharge had large OR for postdischarge 
mortality. In-hospital mortality OR were found to be: 
food, 7.0; neurological, 9.4; musculoskeletal, 6.9; 
safety, 5.6; psychosocial, 6.7; respiratory, 8.1; skin, 
5.2; genitourinary, 3.0; gastrointestinal, 2.3; peripheral- 
vascular, 3.9; cardiac, 2.8; and pain, 1.1. CI at 95% are 
within ±20% of these values, with p<0.001 (except for 
pain). Similar results applied to postdischarge 
mortality. All results were comparable across the two 
1-year periods, with 0.85 intraclass correlation 
coefficient. 

Conclusions: Nursing assessments are strongly 
correlated with in-hospital and postdischarge mortality. 
No multivariate analysis has yet been performed, and 
will be the subject of a future study, thus there may be 
confounding factors. Nonetheless, we conclude that 
these assessments are clinically meaningful and valid. 
Nursing assessment data, which are currently unused, 
may allow physicians to improve patient care. The 
mortality OR and the dynamic nature of nursing 
assessments suggest that nursing assessments are 
sensitive indicators of a patient's condition. While 
these conclusions must remain qualified, pending 
future multivariate analyses, nursing assessment data 



ARTICLE SUMMARY 



Article focus 

■ This study investigates risks of mortality asso- 
ciated with entry and last predischarge nurses' 
assessments of patients' conditions by physio- 
logical system. 

■ It is the first quantitative study of the validity and 
clinical implications of nurses' head-to-toe clin- 
ical assessments. 

Key messages 

Entry nursing assessments (other than pain) are 
strongly correlated with in-hospital mortality, and 
final predischarge nursing assessments (other 
than pain) are strongly correlated with post- 
discharge mortality, independent of diagnosis 
and medical history. It is evident that most 
nursing assessments are clinically meaningful 
and valid. 

The dynamic nature of in-hospital nursing 
assessments and the large mortality OR asso- 
ciated therewith suggest not only are nursing 
assessments sensitive indicators of a patient's 
condition, but they may also aid in detection of 
clinical problems as they develop during the 
course of a patient's stay. 
Nursing assessment data, which are now essen- 
tially unused, provide additional information on 
patients' conditions and should allow physicians 
to improve patient care and reduce in-hospital 
mortality. 



ought to be incorporated in risk-related health 
research, and changes in record-keeping software are 
needed to make this information more accessible. 



BACKGROUND 

Nursing budgets constitute a major part of a 
hospital's operating costs, accounting for 
some 25% of the total operating budget and 
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Clinical implications and validity of nursing assessments 



Strengths and limitations of this study 

■ This is the first quantitative study of the clinical validity of 
nursing assessments. More than 42 000 patient visits over 
two 1 year periods give the study a strong statistical base. 

■ All the OR reported are both statistically and clinically signifi- 
cant, with none of the 95% CI's overlapping one (except for 
pain). 

It is internally consistent, with in-hospital and postdischarge 
OR's for all time periods yielding similar results, providing evi- 
dence of reliability of nursing assessments. 

■ There are some limitations to our study. No multivariate ana- 
lysis was performed, making the associations found subject to 
possible unknown confounders. The work has been done at a 
single site and that site has a population skewed older, which 
raises questions about generalisability. 



44% of direct care costs. 1 2 Recent studies have demon- 
strated that higher staffing ratios of registered nurses are 
associated with fewer hospital-related deaths, failures to 
rescue, cardiac arrests, hospital-acquired pneumonia 
and other adverse events, as well as having positive 
effects on patients' safety in intensive care units and in 
surgical patients. 3-6 Increased registered nurse hours 
spent on direct patient care were associated with 
decreased risk of hospital-related death and shorter 
lengths of stay. 3-6 In the course of providing direct 
patient care, nurses assess each patient by physiological 
system and record those assessments in the electronic 
medical record (EMR) . Once recorded however, nursing 
notes are not often read by attendants or residents. 7 

There have been several studies of the relationship 
between registered nurse staffing and in-hospital mortal- 
ity. 8 9 However, no previous studies have demonstrated a 
direct connection between nursing assessments and 
patient risk of mortality. Furthermore, while the nursing 
literature is replete with studies of nursing diagnostic ter- 
minology and its standardisation, 10-19 there is a paucity 
of quantitative studies of the validity and clinical implica- 
tions of nurses' head-to-toe clinical assessments. In this 
article we investigate clinical associations of 12 simplified 
nursing assessments, one for each physiological systems, 
with both in-hospital and postdischarge mortality data. 
Our hypotheses are that nursing assessments performed 
at entry of in-patients are predictors of in-hospital mor- 
tality, and nursing assessments performed just before 
in-patient discharges are predictors of postdischarge 
mortality. 



METHODS 

This research was initiated in an attempt to understand 
issues of continuity of care in hospitals. What piqued 
our interest in nursing assessments is the fact that other 
than laboratory tests and vital signs, they are the only 
clinical variables in the EMR that are reflective of 
patient condition that are not static, and their values are 
updated regularly. The results of this research are 



incorporated in the 'Rothman Index', which is a new 
measure of patient condition. This is the first founda- 
tional paper in a series of studies related to the scientific 
basis of the Rothman Index. Approval for the work was 
granted by the Sarasota Memorial Hospital Institutional 
Review Board. 

Nursing assessment data for the periods 1/2004-12/ 
2004 and 7/2005-6/2006 were extracted from the 
Electronic Hospital Record at Sarasota Memorial 
Hospital, an 805-bed community hospital. Our cohort 
for this study were all patients admitted for any reason 
during these periods, excluding obstetrics, paediatric 
and psychiatric patients, which determined the study 
size of 42 302 inpatient visits. For the discharge study, we 
had complete data for 39 964 inpatient visits in which 
the patient was discharged alive. Demographic data and 
diagnostic data have not been collected for this popula- 
tion; however, our subject community hospital serves a 
population skewed older than the US average. 

In general, nurses' assessments are entered into the 
Electronic Medical Record in one of two ways: either the 
nurse answers a series of detailed questions to document 
each assessment, such as for respiratory, 'What are the 
breath sounds?', 'What colour are the nail beds', etc; or, 
the 'charting by exception' method is used, where the 
nurse records a simple overall answer of 'met' or 'not 
met' for each physiological system, such as 'Does the 
patient's respiratory function meet a minimum stand- 
ard?' (if the answer is 'not met', only then are detailed 
follow-on questions considered). In this study the binary 
charting by exception method was used, and the assess- 
ments were characterised as either having 'met' the 
standard or having 'not met' the standard in each of the 
following 12 areas: food, neurological, safety, skin, 
genitourinary, musculoskeletal, respiratory, cardiac, per- 
ipheral vascular, gastrointestinal, psychosocial and pain. 
Definitions of the relevant standards are shown in the 
appendix. Nursing assessments were generally per- 
formed at least once per shift. For each area of nursing 
assessment, the all-cause in-hospital death rates and mor- 
tality OR's, associated with patients' entry nursing assess- 
ments, were computed and the all-cause death rates and 
mortality OR's, associated with patients' last assessments 
prior to discharge, were computed for patients living at 
the time of discharge with deaths within the time 
periods 2 days, 30 days and 1 year from discharge. 
Mortality was established by comparison with the Social 
Security Administration Death File. The reproducibility 
of outcome measures between the 2004 data and the 
2005-2006 data was assessed using an intraclass correl- 
ation coefficient. The data analysis was carried out utilis- 
ing SAS V.9.2 (SAS Institute, Gary, North Carolina, USA) 
and Systat V.13 (Systat Corp., Chicago, Illinois, USA). 



RESULTS 

The population studied had a mean of 4.7 days and 
median 3.1 days length of stay, with an SD of 5.1 days. 
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Table 1 Nursing assessment data with resulting 30-day mortality OR 



Nursing 


Mat 

iviei 


iviei 


iviei moriciMiy 


IMOI ITlcl 


imoi met 


imoi rnei monaiiiy 






assessment 


live 


died 


odds 


live 


died 


odds 


Urf 


f% If nil 1 A 

p value 


Food 


34769 


705 


0.020 


3383 


1084 


0.320 


16 


<0.001 


Neurological 


34600 


770 


0.022 


3561 


1018 


0.286 


13 


<0.001 


Psychosocial 


36327 


1260 


0.035 


1834 


525 


0.286 


8.3 


<0.001 


Safety 


32449 


781 


0.024 


5157 


993 


0.193 


8.0 


<0.001 


Genitourinary 


34214 


1110 


0.032 


3926 


679 


0.173 


5.3 


<0.001 


Skin 


29627 


585 


0.020 


8506 


1199 


0.141 


7.1 


<0.001 


Musculoskeletal 


24528 


243 


0.010 


13630 


1546 


0.113 


11 


<0.001 


Respiratory 


26941 


449 


0.017 


11223 


1340 


0.119 


7.2 


<0.001 


Gastrointestinal 


32365 


1098 


0.034 


5797 


690 


0.119 


3.5 


<0.001 


Peripheral vascular 


28914 


828 


0.029 


9240 


961 


0.104 


3.6 


<0.001 


Cardiac 


31947 


1119 


0.035 


6228 


670 


0.108 


3.1 


<0.001 


Pain 


33618 


1568 


0.047 


4436 


218 


0.049 


1.1 


0.474 



Numbers of patients dead and living at 30 days from date of discharge who were denoted 'met' 
their associated all-cause mortality odds and OR's. The 95% CI's for the OR's are all less than 
one; p values for OR's are estimated by the Fisher Exact test (two-tailed), and listed in the last 
0.001 is for the pain assessment. 



or 'not met' at last in-hospital assessment, and 
±15% of the values given and none overlaps 
column. Note the only p value larger than 



An example of the data and associated results for 
30-days postdischarge mortality is given in table 1, illus- 
trating our calculations. Results for in-hospital mortality 
OR's associated with failing an entry nursing assessment 
are given in table 2 for 42 302 patients of whom 1086 
died in the hospital. OR's for deaths within post- 
discharge periods of 2 days, 30 days and 1 year for 
39 964 patients for whom we had data and who were 
living at the time of discharge are given in table 3. 
There were less than 0.3% missing data for any result. 
Although generally, about 90% of patients passed the 
assessments, in all categories and for both in-hospital 
and postdischarge deaths, not meeting standards for an 
assessment resulted in significantly higher death rates 
than meeting standards and very large mortality OR's, 
with the single exception of the pain assessment. Except 
for the pain assessments, all results are statistically signifi- 
cant (p<0.001) and none of the 95% CI's overlap one. 
The implications of pain assessments are examined in 
the discussion section below. 



Table 2 In-hospital death OR associated with entry 



Nursing assessment 


OR 


95% CI 




P 


Neurological 


9.4 


8.3 


10.6 


<0.001 


Respiratory 


8.1 


7.0 


9.3 


<0.001 


Food 


7.0 


6.1 


7.9 


<0.001 


Musculoskeletal 


6.9 


5.9 


8.1 


<0.001 


Psychosocial 


6.7 


5.9 


7.7 


<0.001 


Safety 


5.6 


4.9 


6.3 


<0.001 


Skin 


5.2 


4.6 


5.9 


<0.001 


Peripheral vascular 


3.9 


3.5 


4.4 


<0.001 


Genitourinary 


3.0 


2.6 


3.4 


<0.001 


Cardiac 


2.8 


2.5 


3.2 


<0.001 


Gastrointestinal 


2.3 


2.0 


2.5 


<0.001 


Pain 


1.1 


0.9 


1.2 


0.530 





To evaluate the agreement between 2004 and 2005- 
2006 values, first we calculated all the OR's for the 2004 
and 2005-2006 subsets of data, and then an intraclass 
correlation coefficient was calculated comparing all 
OR's for 2004 to their counterparts from 2005 to 2006 
across all categories and all time points (ie, in-hospital 
and 2, 30, and 365 days postdischarge) and found to be 
0.85; values greater than 0.75 indicate excellent reprodu- 
cibility. 20 Thus, there is excellent reproducibility 
between the results for 2004 and those for 2005-2006 
across all our measures of interest. 



Table 3 Postdischarge mortality OR for final nursing 


assessments 










Deaths within postdischarge 




period 






Nursing assessment 


2-day OR 


30-day OR 


1-year OR 


Food 


37 


16 


6.7 


Musculoskeletal 


28 


11 


4.6 


Neurological 


27 


13 


6.5 


Psychosocial 


15 


8.3 


5.3 


Respiratory 


13 


7.2 


4.2 


Safety 


13 


8.0 


5.0 


Skin 


10 


7.1 


4.3 


Genitourinary 


8.4 


5.3 


g 


Peripheral vascular 


5.9 


3.6 




Cardiac 


5.4 


3.1 


2.3 


Gastrointestinal 


4.6 


3.5 


2.2 


Pain 


2.2 


1.1 


0.8 


All-cause mortality OR for postdischarge deaths for periods of 


2 days, 30 days and 1 year from the date of hospital discharge 


associated with last in-hospital assessments prior to discharge. 


The 95% CI for the OR's are less than 


±10% (±15%, 


±35%) of the 


values for the 1 year (30 day and 2 day) postdischarge values 


shown, and none overlaps the value 1 except for pain; p<0.001 for 


all OR's except for pain. 
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DISCUSSION 

There is a growing body of evidence that suggests that 
nurses have a separate and identifiable effect on patient 
hospital outcomes, irrespective of other medical care (cf 
Kane et af 1 22 and references cited therein). An import- 
ant subject area in the literature is whether nursing data 
collected during the hospital stay can be used to explain 
commonly studied variables such as costs/charges, 
lengths of stay and mortality. However, until now, 
nursing assessments have not been shown to have clin- 
ical predictive validity. Our study combines over 40 000 
cases, encompassing all admissions for any reason 
except for paediatrics, psychiatric and rehab, and we 
have shown that patients failing to meet minimum stan- 
dards for any nursing assessment had significantiy 
higher in-hospital and postdischarge death rates than 
patients who did meet these standards, regardless of 
medical history and diagnosis. This is evidence of the 
clinical validity of nursing assessments. In fact, the 
medical literature is replete with studies of critically ill 
patients for whom variables with mortality OR in the 
range 2-4 are considered sensitive and important pre- 
dictors of mortality. 23-26 The OR's we report demon- 
strate that these measures of a patient's functionality are 
significant and sensitive indicators of a patient's condi- 
tion. Thus, nursing assessments may aid in physician 
care and possibly reduce hospital patient mortality, and 
future risk-related health research ought to consider 
incorporation of nursing assessments. Further study is 
clearly warranted. 

We have studied deaths without regard to diagnosis, age, 
gender or severity of illness that occur in-hospital as well as 
within short (2 and 30 days) and long (1-year) periods of 
time for two reasons: (1) in order to learn whether there 
might be similar or different short and long-term relation- 
ships of assessments to mortality and (2) as a way to test val- 
idity and reliability. We have found that failing a nurse's 
assessment has both short-term and long-term correlations 
with mortality. The fact that our results are consistent over 
short and long periods, over all assessments, and from a 
medically diverse patient population, demonstrates that 
nursing assessments have valid and significant clinical 
implications irrespective of medical histories and diagno- 
ses. While we have not compared survivor demographics 
with those who died, it is unlikely that such large OR's 
would be explained by underlying demographic factors 
like age or race or gender. 

One might ask whether the high OR's associated with 
these assessments may simply represent high-risk diagno- 
ses. We have not built a multiple linear regression model 
to determine the added predictive power of nursing 
assessments to other indicators of mortality. However, 
this is an area of current investigation. Preliminary 
results, which we shall publish in subsequent studies, 
show that among patients who were admitted with a 
diagnosis of congestive heart failure, those who failed 
any of the food, neurological, musculoskeletal or skin 
nursing assessments at admission had a mortality OR of 



6-9, versus those who passed. Further, failing multiple 
nursing assessments had a greater implication in terms 
of in-hospital mortality than failing just one. Generally, 
patients who fail nursing assessments may well be those 
who also have serious or even terminal diagnoses. 
Nonetheless, here we have demonstrated that nursing 
assessments at both entry and discharge capture the ser- 
iousness of the patient's condition irrespective of diag- 
nosis. Given the size of the study, it is clear that these 
variables are meaningful indicators of both in-hospital 
and postdischarge mortality. 

The highest in-hospital mortality OR's, 7 or greater, 
are associated with those patients not passing the food, 
neurological, psychosocial or musculoskeletal assess- 
ments. It is important to comprehend the simple clarity 
of these findings. For example, if a newly admitted 
patient is not able to move independendy in his/her 
bed, or behaves inappropriately, or cannot chew and 
swallow food — that patient is almost seven times more 
likely to die in the hospital than patients who have none 
of these problems. If a patient is incoherent or is not 
oriented, that the patient is almost 10 times more likely 
to die in the hospital than those who are coherent and 
alert, no matter what the diagnosis may be. In contradis- 
tinction, pain is not a significant indicator of mortality. 
This may be because pain, unlike the other problems 
indicated by nursing assessments, can often be con- 
trolled independent of the patient's general condition, 
so one does not expect it to have similar clinical implica- 
tions. Further studies may clarify this issue. 

Since we have shown that initial and final nursing 
assessments contain clinical information, it is reasonable 
to infer that all nursing assessments (other than pain) 
gathered throughout the patient's stay in the hospital 
contain significant clinical information. Nursing assess- 
ments are generally recorded every 12 h, and sometimes 
more frequently, throughout the patient's stay. 
Therefore, these data represent a changing indicator of 
patient condition. Thus, the importance of this work is 
in alerting physicians to a source of longitudinal clinical 
information about the patient's condition that no static 
measurement, such as demographics or principal diag- 
nosis, can provide and which is not currendy being rou- 
tinely utilised. 

Although our results show excellent reproducibility 
across two Tyear time periods, we are aware of questions 
about the reliability of nursing data, specifically 
intra-rater and inter-rater reliability. However, reprodu- 
cible and consistent results such as shown here are only 
possible with reproducible and consistent nursing assess- 
ments. One has to conclude that by and large, the 
nurses get it right, and that they provide an important 
and valuable tool for assessing patient condition, irre- 
spective of medical diagnosis and history. Thus, it is sug- 
gested that hospital physicians make special effort to 
ascertain whether patients have passed or failed their 
nursing assessments, a practice not widely followed cur- 
rendy by attendants or residents. 7 
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We have established that simplified nursing assess- 
ments, gathered throughout a patient's stay and which 
are now essentially unused by physicians, have clinical val- 
idity. If physicians were to utilise these data, they then 
would be adding an important and largely 'new' source 
of clinical information to their evaluations. For example, 
when a physician sees a patient in the hospital, he or she 
often consults with the bedside nurse. However, this con- 
sultation is not always possible or practical. In many cases, 
consulting with the nurse from the previous shift, or the 
previous day, or even the day before that, to understand 
changes in the patient's condition, is not feasible. And 
even if it were, getting a verbal report on a patient's con- 
dition about the previous several days would likely be 
incomplete. These circumstances make use of nursing 
assessments, recorded in the EMR, the only effective way 
to gain access to this clinically relevant information. And 
though immediate observations are the most important 
in determining care, the prior observations provide an 
important and meaningful context, allowing the phys- 
ician to assess the changes in patient condition. It must 
be noted however, that current EMR technology does not 
facilitate quick and easy physician access to nurses' obser- 
vations and assessments. Either changes in current 
record-keeping software or adjuncts to it would make 
access to this information more accessible. 

CONCLUSIONS 

Entry nursing assessments of in-hospital patients are 
strongly associated with in-hospital mortality, and predis- 
charge nursing assessments of in-hospital patients are 
associated with postdischarge mortality. No multivariate 
analysis has yet been performed, and will be the subject 
of a future study, and thus there may be confounding 
factors. However, it is difficult to hypothesise any alterna- 
tive factors that might confound our results, given the 
high OR's and the variety, multiple time intervals, and 
extent of data considered. We infer that these nursing 
assessments have valid and significant clinical meaning 
irrespective of medical histories and diagnoses. It is rea- 
sonable to infer further that nursing assessments taken 
throughout a patient's stay are also clinically meaningful. 
These assessments, which are part of what is termed as 
the 'head-to-toe' patient assessment, and which are a 
standard part of nursing school curricula, are collected 
and recorded at all hospitals, and simplified summaries 
of assessments, as we have analysed, can be constructed. 
Since nursing assessments are recorded at least every 
12 h throughout the patient's stay, they represent a chan- 
ging indicator of patient condition. Thus, they make 
available real-time longitudinal sensitivity that no static 
measurement, such as demographics or principal diag- 
nosis, can provide. 

The large OR's suggest that nursing assessments are 
sensitive indicators of clinical problems during the 
course of a patient's hospital stay. This compact clinical 
data source in the EMR is a natural longitudinal source 



of information, providing physicians access to the 
insights of nurses as recorded throughout the patient's 
entire stay. Such dynamic information should allow phy- 
sicians to improve patient care. While these conclusions 
must remain tentative, pending detailed multivariate 
analyses, we believe nursing assessment data ought to be 
incorporated along with standard diagnoses in future 
risk-related health research. Current EMR technology 
does not allow quick and easy access to nurses' observa- 
tions and assessments, so changes in current record- 
keeping software or adjuncts to it will be necessary to 
make this information more accessible. 
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APPENDIX 

Nursing assessments 

The nursing assessments are generally performed at least once per 
shift. When documented as described above, they consist of binary 
data, characterised either as having 'met' the standard or having 'not 
met' the standard. For each physiological system, definitions for the 
relevant standards at Sarasota Memorial Hospital are shown below. 
Although standards vary across hospitals, basically the same data are 
being collected at every hospital. It is generally possible to construct 
similar binary variables from any hospital's nursing data. 

Cardiac standard: Pulse regular, rate 60-100 bpm, skin warm and dry. 
Blood Pressure less than 140/90 and no symptoms of hypotension. 

Food /nutrition standard: No difficulty with chewing, swallowing or 
manual dexterity. Patient consuming >50% of daily diet ordered as 
observed or stated. 

Gastrointestinal standard: Abdomen soft and non-tender. Bowel 
sounds present. No nausea or vomiting. Continent. Bowel pattern 
normal as observed or stated. 

Genitourinary standard: Voids without difficulty. Continent. Urine 
clear, yellow to amber as observed or stated. Urinary catheter patent if 
present. 

Musculoskeletal standard: Independently able to move all extremities 
and perform functional activities as observed or stated (includes assist- 
ive devices) . 

Pain standard: Without pain or visual analogue scale<4 or experien- 
cing chronic pain that is managed effectively. 

Neurological standard: Alert, oriented to person, place, time and situ- 
ation. Speech is coherent. 

Peripheral/vascular standard: Extremities are normal or pink and 
warm. Peripheral pulses palpable. Capillary refill <3 s. No oedema, 
numbness or tingling. 

Psychosocial standard: Behaviour appropriate to situation. Expressed 
concerns and fears being addressed. Adequate support system. 

Ftespiratmy standard: Respiration 12-24/min at rest quiet and regular. 
Bilateral breath sounds clear. Nail beds and mucous membranes pink. 
Sputum clear if present. 

Safety /fall risk standard: Safety/fall risk factors not present. Patient is 
not a risk to self or others. 

Skin/tissue standard: Skin clean, dry and intact with no reddened 
areas. Patient is alert, cooperative and able to reposition self independ- 
ently. Braden scale>15. 



6 



Rothman MJ, Solinger AB, Rothman SI, etal. BMJ Open 201 2;00:e000849. doi:1 0.1 136/bmjopen-201 2-000849 



